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1.  INTRODUCTION 

The  aim  of  this  paper  is  to  consider  what  advice  a  statistician  should 
give  in  the  following  situation. 

Example.  In  order  to  detect  a  possible  difference  in  performance  between  two 
types  of  subject,  a  controlled  comparison  is  carried  out  four  times  and  the 
significance  levels  observed  in  independent,  one-sided  tests  are  .04,  .15,  .01, 
.05.  The  investigator  consults  a  statisticiain  about  the  proper  combination  of 
these  results  and  is  recommended  to  use  Fisher’s  method,  based  on  the  product 
of  the  observed  levels.  He  does  this  correctly  amd  obtains  a  nominal 
significance  level  a^=. 00131.  However,  the  investigator  then  reveals  that  he 
is  uneasy  about  the  method  because  the  number  of  comi>arisons  was  not  fixed  in 
advance.  He  simply  continued  the  sequence  of  experiments  until  he  thought 
there  was  enough  evidence  to  reject  the  null  hypothesis  and  establish  a  real 
difference  between  the  two  types  of  subject. 

In  his  first  reaction,  the  statistician  might  well  regret  hearing  the  last 
remark  since  it  weakens,  if  not  destroys  the  foundations  of  his  previous 
advice.  On  the  other  hand,  perhaps  the  situation  illustrated  by  this  example 
is  common  enough  to  deserve  a  more  serious  response.  Caun  we  properly  say 
anything  about  the  combined  significance  of  a  sequence  of  experiments  when  the 
stopping  rule  is  ill-defined? 

Before  we  attempt  to  answer  this  question,  let  us  consider  a  more  general 

setting.  Suppose  that  a  sequence  of  m  independent  experiments  is  carried  out 

and  that  the  results  are  used  separately  to  test  a  certain  null  hypothesis  Hq, 

the  same  test  statistic  t  being  used  in  each  case.  For  convenience,  let  us 

assume  that  t  is  a  continuous  variable  with  a  distribution  function  <I>  ,  under 

o 

Hq,  such  that  the  observed  level  of  significance  u  =  1  -  ^^(t)  is  uniformly 

distributed  on  the  interval  [0.1].  Thus,  we  are  assuming  that  the  detailed 

results  of  the  original  experiments  have  been  reduced  to  a  sequence  of  observed 

significance  levels  u, ,u„ . u  and  that,  under  the  null  hypothesis,  this  is 

l  di  m 
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equivalent  to  a  random  sample  from  the  uniform  distribution  on  [0,1]. 

For  a  single  experiment,  rejecting  Hq  when  t  is  large  is  equivalent  to 
using  u=l-4^(t)  as  a  test  statistic  and  there  is  an  implicit  ordering  of  the 
underlying  sample  sp>ace  in  which  points  are  treated  as  "extreme"  if  the 
corresponding  value  of  u  is  small.  A  justification  for  this  might  be  an 
application  of  the  Neyman-Pearson  Lemma,  for  some  alternative  hypothesis.  For 
example,  suppose  such  a  hypothesis  restricts  the  distribution  of  u  to  a  family 
of  Lehmann  alternatives  with  probability  densities  of  the  form  : 

h(u)  =  eu®"\  0  <  u  <  1,  (1) 

with  parameter  0,  0  <  6  <  1.  Then,  rejection  of  when  the  observed  level  u 
is  small  is  justified  by  Neyman-Pearson  theory.  Of  course,  the  same  is  true 
for  any  alternative  density  h,  provided  that  h(u)  is  decreasing  in  u  over  the 
unit  interval.  However,  it  turns  out  that  the  family  of  Lehmann  alternatives 
is  particularly  appropriate  for  dealing  with  combinations  of  several  observed 
levels. 

Suppose  m  is  fixed,  m  >  2.  leaving  aside  the  question  of  stopping  rules 
for  the  moment.  We  are  tacitly  assuming  that  the  experiments  are  all  similar 


and  have  similar  sample  sizes,  so  it  is  natural  to  give  equal  weights  to  the 

components  Uj.U2 . u^  in  assessing  their  combined  signif icemce.  The  usual 

method  of  combining  m  independent  tests  is  based  on  the  product 

2 

V  =  u,u~  ...  u  and  it  relies  on  the  fact  that  -21og  v  has  a  v  -distribution 
m  1  2  m  ®  m 

with  2m  degrees  of  freedom.  The  method  is  due  to  Fisher  (1950)  and  it  is  also 


described  in  steindard  texts  such  as  Cox  and  Hinkley  {'.974),  see  page  80.  For 

n 

our  purpose,  it  will  be  more  convenient  to  write  S  =  2  w,,  where  w.  =  -log  u.. 

n  j  1  1  1 

Thus,  (S^.n^l)  is  a  random  ¥ralk  and,  under  the  null  hypothesis,  the  independent 
steps  w^  have  a  common  probability  density  e  ,  w>0.  In  p>articular,  it  is 
easily  verified  that 


a 

o 


(2) 
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In  the  above  example,  m=4  and  s  =  12.72,  so  the  nominal  significance  level  is 

=  .00131.  It  will  be  argued  later  that,  in  the  absence  of  a  stopping  rule. 

a  lower  bound  on  reasonable  significance  levels  is  a  =.00458  :  see  equation 

(16).  More  precisely,  a  =  min{a{0),  O<0<1},  where  0  represents  a  simple 

alternative  hypothesis  and  a(0)  is  determined  by  stopping  the  process  {S^}  as 

soon  as  the  likelihood  ratio  exceeds  a  certain  critical  value. 

The  main  argument  of  this  paper  relies  on  several  principles.  The  first 

is  to  use  Fisher’s  method  of  combining  significance  levels  when  the  number  is 

.xed  in  advance.  Other  methods  have  been  suggested  in  the  literature,  for 

example,  several  are  mentioned  in  a  discussion  of  asymptotic  optimality  by  Berk 

autd  Cohen  (1979).  However,  they  do  not  seem  appropriate  here.  The  second 

principle  is  that  the  ordering  of  points  (Uj,U2 . u^)  in  IR*”  determined  by  the 

product  V  can  be  formally  justified  by  an  application  of  the  Neyman-Pearson 
in 

Lemma,  for  some  alternative  hypothesis.  We  postulate  an  alternative  Hj ,  which 

may  be  composite  and  assume  that  the  observed  levels  u, .u., . u  are 

I  m 

independent  and  identically  distributed  not  only  under  the  null  hypothesis  Hq, 
but  also  under  .  Then,  as  we  shall  see  in  Section  2,  the  only  alternative 
distributions  of  u  with  the  required  properties  are  those  given  by  (1)-  The 
final  principle  is  one  of  conservatism  in  the  evaluation  of  a(0).  When  the 
number  of  exp>eriments  Is  not  prescribed  and  a  range  of  values  is  jxjssible,  the 
simple  alternatives  represented  by  0,  0  <  0  <  1,  lead  to  different  orderings  of 
the  corresponding  sample  space.  If  there  is  no  clearly  defined  stopping  rule, 
then  it  seems  reasonable  to  consider  one  that  maj-.imizes  the  error  probability 
a(0)  and  this  corresponds  to  using  a  test  of  power  one. 

The  next  section  gives  a  version  of  the  Neyman-Pearson  Lemma  for 
independent  sampling  up  to  a  well-defined  stopping  time.  This  is  followed  by  a 
discussion  of  alternative  hypotheses  for  our  particular  problem.  The  explicit 
calculation  of  a(0)  is  described  in  Section  3  and  it  is  a  simple  matter  to  find 
the  minimum 
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a**  =  min{a(9).  0  <  0  <  1}.  (3) 

In  general,  the  results  u,.u„.....u  of  m  tests  determine  a  final  value  S  =  s 
®  12m  m 

and  the  corresponding  levels  and  a  depend  only  on  m  and  s,  with 
0  <  <  a  <1.  The  interpretation  of  as  an  observed  significance  level  is 

familiar  and.  provided  that  m  is  fixed,  it  can  be  justified  in  terms  of 
Neyman-Pearson  theory  by  appealing  to  any  one  of  the  alternative  hypotheses 
represented  by  0.  0  <  9  <  1.  In  the  absence  of  any  stopping  rule,  a  is  a 
useful  measure,  but  its  interpretation  is  more  complex,  since  each  value  of  6 
leads  to  a  different  ordering  of  the  set  of  possible  stopped  sequences. 
Unfortunately,  the  choice  of  9  is  arbitrary  and  all  we  can  say  is  that  a  is 
the  smallest  significance  level  that  can  be  justified  by  applying  the 
Neyman-Pearson  Lemma,  for  some  alternative  hypothesis.  Roughly  sp)eaking.  the 

if 

ratio  a  /a^  is  a  measure  of  the  effect  of  failing  to  specify  the  stopping  rule 
in  advance.  The  final  section  gives  some  numerical  values  of  the  ratio, 
corresponding  to  nominal  significance  levels  =  .01  and  .001. 

2.  PRELIMINARIES 

Let  y  =  (yj,y2'- •■)  ^  sequence  of  i.i.d.  random  variables  and  suppose 

there  are  two  simple  hypotheses,  Hq  aind  Hj ,  which  specify  their  common 

probability  density  as  f  or  g.  respectively.  Suppose  further  that  the  stopping 

rule  is  given  and,  for  convenience,  let  this  be  determined  by  continuation  sets 

Cj,C2 .  where  each  is  a  measurable  subset  of  For  any  sequence  y,  let 

y^  =  (y, .y^ . y  )■  It  is  assumed  that,  if  y^  e  C  ,  then  y"*  t  C  for  m  = 

1  ^  n  n  m 

1.2 . n-1.  The  stopping  time  is 

N{y)  =  min{n>l:  y^  C  C^} .  (4) 

so  that  N(y)  =  «>  if  y^  e  for  every  n  ^  1.  Consider  the  stopping  set  at  time 
n  : 

D  =  {y^  t  IR^  :  y™  t  C  ,  m<n.  y"^  €  C  }. 
n  '  m  ■'  n' 

For  y*'  £  D^,  the  observed  sequence  that  arises  from  stopping  will  be  written 
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(n.y^ ,y2. . . .y^^) •  A  terminal  decision  rule  is  defined  by  any  measurable 

partitions  of  the  stopping  sets  :  D  =  A  U  B  ,  n=l,2 .  Thus,  H„  is 

n  n  n  U 

accepted  if  N(y)  =  n  and  t  A^.  for  some  n  >  1,  and  rejected  if  N(y)  =  n  and 
y^  £  We  are  concerned  here  with  comparing  terminal  decisions  for  a  given 

stopping  rule  and  finite  stopping  times,  so  the  case  N(y)  =  ®  can  be  treated 
arbitrarily.  For  simplicity,  suppose  that  is  always  accepted  if  N(y)  = 
Then  the  error  probabilities  are 


a  =  2  dP..  p  =  2  J,  dP  +  P  (N 

^  f  ,  A  g  g  ' 


(5) 


n=l  n 


n=l  n 


where  P.  and  P  refer  to  the  distributions  of  y  under  H„  and  H, .  Now  consider 
f  g  ■'0  1 

the  stopped  likelihood  ratio  when  N  =  n.  For  an  observed  sequence  (n, 


.y2>  •  likelihood  ratio  is 


dP 


dP, 


=  x„{y  )  =  n 


n  g(y,) 


i:i  '(^i)  ■ 


(6) 


Def ini t ion .  We  say  that  the  partitions  (A^.B^.  n>l}  define  a  likelihood  ratio 
test  if  there  is  a  positive  constant  X  such  that  Xj^(y^)  <  X  if  y^  £  A^  and 

\(y")  >  A  if  y^  £  B^.  n  =  1.2 . 

Note  that  the  standard  proof  of  the  Neyman- Pear son  Lemma  applies  to  two 
measures  on  any  a-field  and.  in  particular,  to  a  stopped  o-field.  This  leads 
to  the  result  that,  in  general,  likelihood  ratio  tests  have  the  following 
optimality  property. 


Lemma.  Let  fA  .B  )  define  a  likelihood  ratio  test  with  critical  value  X  >  0 
-  '  n  n' 

and  error  probabilities  a.P  given  by  (5).  Oanslder  emy  other  terminal  decision 
rule  defined  by  pjartitions 

D  =  A'  U  B' .  n=l .2. . . .  and  let  a' ,P'  be  the  corresponding  error  probabi 1 i ties 
n  n  n  t-  =>  r- 

Then. 

a'  <  a  =>  P'  >  P  and  a’  <  a  =>  P'  >  p. 


The  lemma  shows  that,  for  any  stopping  rule,  the  ordering  of  observed 
sequences  should  be  based  on  the  likelihood  ratio  and  this  holds  whether  we  are 
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comparing  data  vectors  of  the  same  or  different  dimensions. 

Before  we  can  apply  this  result,  we  need  to  discuss  alternative  hypotheses 

for  the  problem  of  combining  independent  significance  tests.  In  the  first 

place,  let  us  return  to  the  case  when  the  number  of  tests  is  fixed.  There  is 

no  given  alternative,  but  it  is  difficult  to  justify  any  method  of  combination 

and  the  implicit  ordering  of  vectors  of  significance  levels  without  introducing 

one.  In  fact,  there  are  many  alternatives  consistent  with  the  usual  method  of 

combination.  For  a  fixed  number  of  tests  with  observed  levels  Uj,U2 . ti^, 

m  >  2.  the  ordering  based  on  the  product  v  =  u,u„...u  is  equivalent  to  using 
m 

the  statistic  2  w.  on  the  set  of  points  {(w, .w_ . w  ):  w  >0.  1  <  i<  m} .  In 

1  '  1  Z  m  l 

i 

m  m 

other  words,  (wj.  w^ . w^)  is  more  extreme  than  (Wj , w^. . . ■ . w^)  if  2  wj  >2 

Wj .  Under  the  null  hypothesis,  when  the  u.  are  uniformly  distributed  on  [0.1], 
we  have  the  probability  density  f{w)  =  e  w  >  0,  for  the  w.  =  -log  u..  Now 
consider  an  alternative  hypothesis  under  which  their  common  probability  density 
is  g(w)  and  assume  that  g(w)  >  0  and  its  derivative  g'(w)  is  continuous  for 

w  >  0.  The  likelihood  ratio  is  given  by 

m  m 

log  ■  _  =  2  log  g(w  )  +  2  w 
mi  i  1  ‘ 

m 

It  can  only  produce  the  same  ordering  as  2  w.  if  it  is  a  function  of  this  sum. 

1  ^ 

m  m 

Then,  differentiation  shows  that  2  d(log  g(w.))  =  0  whenever  2  dw  =  0.  Since 

1  ^  1 

m  >  2,  it  follows  that  g'(w^)/g(wp  is  constEUit  for  all  w^  >  0  and,  hence,  the 
alternative  hypothesis  must  be  represented  by  an  exponential  distribution  : 
g(w)  =  0e  ,  w  >  0.  (7) 

m 

The  parameter  0  is  positive  and,  since  X  is  required  to  be  increasing  in  2  w  , 

m  ^  ^ 

we  demand  that  0  <  0  <  1.  Finally,  it  is  easily  verified  that  (7)  is 

equivalent  to  the  family  of  Lehmarm  alternatives  (1).  for  the  original 

significance  levels  u^. 
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3.  SIGNIFICANCE  LEVELS 

We  now  turn  to  the  question  of  assessing  the  significance  attained  by  a 
series  of  tests  when  there  is  no  clearly  defined  stopping  rule.  The  null 
hypothesis  is  given  by  (7).  with  0  =  1.  and  we  first  consider  a  simple 
alternative  corresponding  to  a  fixed  value  of  the  {jarameter  0,  0  <  0  <  1. 
Bearing  in  mind  that  the  investigator  might  have  been  seeking  to  reject  the 
null  hypothesis  as  soon  as  he  felt  convinced  that  this  was  proper,  we  ought  to 
use  a  conservative  evaluation  of  the  "observed”  signi f icamce  level  a(6) .  In 
general,  the  likelihood  ratio  at  any  stage  is 

X  (0)  =  0"exp{(l-0)2  w  }.  (8) 

1 

Suppose  we  are  given  the  results  of  m  tests  and  let  final 

value  of  the  likelihood  ratio.  The  stopping  rule  underlying  these  results  is 

unknown  and,  perhaps,  there  were  intermediate  values  of  the  likelihood  ratio 

exceeding  •  However,  in  view  of  the  lemma  in  Section  2,  any  other  series 

of  tests  must  be  treated  as  stronger  evidence  against  Hq  if  and  only  if  it 

produces  a  final  value  ^^^(9)  ^  ^obs '  most  conservative  evaluation  of  a(0) 

corresponds  to  rejecting  Hq  for  any  sequence  {X^(0)}  such  that  >  ^obs 

some  n  >  1.  This  suggests  a  stopping  rule:  any  sequence  with  sup  X  (0)  > 

n>l 

is  stopped  when  it  first  exceeds  the  critical  level.  The  terminal  decision  is 

to  reject  Hq  whenever  the  stopping  time  is  finite.  Note  that  the  procedure 

depends  on  0.  It  leads  to  a  significance  level  a(0)  which  is  a  msiximum  amongst 

likelihood  ratio  tests  with  the  same  critical  level. 

In  order  to  evaluate  a(0),  consider  the  stochastic  process  .  where 

n 

=  0  and  S  =  2  w.,  n  >  1.  We  note  that  the  condition  X  (0)  >  X  ,  is 
n  ^  1  ■  n'  '  obs 

equivalent  to 

S  >  c  +  nk, 
n  ~ 

c  =  (I  -  9)-'log 


k  =  -(1  -  0)~^log  0. 


(9) 

(10) 
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The  stopping  time  is  determined  by  a  linear  boundary  : 

N  =  min{n  >  1:  >  c  +  nk}.  (11) 

The  likelihood  ratio  test  with  critical  value  X  =  ^q^s  leads  to 

rejection  of  Hq  if  N  is  finite,  so  the  corresponding  error  probability  is 
a{e)  =  Pq(N<«>)  =  Pq{S^  >  c  +  nk  for  some  n  >  1).  (12) 

Any  other  stopping  rule,  followed  by  a  likelihood  ratio  test  with  the  same 
critical  value  must  produce  a  smaller  error  probability  under  This 

is  a  consequence  of  the  definition  of  N  and  the  terminal  decision  rule.  It  is 
a  straightforward  matter  to  evaluate  a(0). 

Proposi tion.  Let  N  be  the  stopping  time  defined  by  (10)  and  (11).  for  a  fixed 
value  of  the  parameter  0.  0  <  0  <  1,  and  suppwase  that  X^^^  >  0.  Then  the 
likelihood  ratio  test  with  this  critical  value  has  error  probabilities 

a(0)  =  ex'bs-  =  0- 

Proof ■  These  results  are  consequences  of  well-known  properties  of  the  random 
walk  see.  for  example,  problem  2.1  in  Siegmund’s  monograph  (1985).  We 

first  show  that  P(0)  =  0.  Under  the  alternative  hypothesis.  {S^}  has 
independent  steps  with  mean  0  ^  and  it  is  easily  verified  from  (10)  that  the 
slope  k=k(0)  has  1  <  k(0)  <  0  It  follows  from  the  strong  law  of  large 

numbers  that  N  is  finite  with  probability  1  and,  according  to  (5).  P(0)  =  0. 

The  formula  for  a(0)  can  now  be  established  by  using  equations  (5)  and 
(6).  We  note  that  P-  and  P  are  both  derived  from  exponential  distributions 

f  g 

and  that 

D  =  B  =  ({w,.w„ . w  ):  S.<c+jk.  I<j<n.  S  >c+nk}. 

nn  l2  nj  n 

CO  CO 

0(0)  =  2  Jg  dPj.  =  2  J-g  (X^(0))~^dP 

n=l  n  n=l  n  ^ 

By  using  (8)  and  (10).  we  find  that 

(X^{0))-'  =  X'bs  exp(-(l-0)(S^-c-nk)). 

On  the  set  B.S  -c-nk=C  is  the  overshoot  beyond  the  linear  boundary 
n  n  n 

when  the  random  walk  stops.  Thus. 
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n=l  n  ** 

Since  (5(6)  =  0,  we  know  that  Pg(N  <  “)  =  1  and  the  above  series  reduces  lo  tfie 

expectation  of  exp{-{ 1-6)C^}  under  the  alternative  hypothesis.  This  is  easy  ttj 

evaluate  for  a  random  walk  with  exponentially  distributed  steps,  since  the 

overshoot  must  also  have  the  probability  density  (7).  The  only  exception  is 

when  X  ,  =9  and,  hence,  c  +  k  <  0.  In  this  case,  the  event  S,  >  c  +  k  always 

occurs  and  Pj.(N=l)  =  ^^(^=1)  =  1  ,  so  a(0)  =  1.  Otherwise,  X^^^^  >  0  and  the 

required  expectation  is 

/"  exp{-(i-0)(:}g(c)d(:  =  9. 

It  follows  that  a(0)  =  ®^obs '  required. 

The  critical  level  of  the  likelihood  ratio  was  defined  as  the  final  value 

observed  in  a  particular  series  of  m  tests.  Let  this  correspond  to  stopping 

the  random  walk  when  =  s ,  so  that  =  S*"  ®59P{(l~0)s}  by  equation  (8). 

The  main  result  of  the  proposition  can  be  expressed  in  a  more  convenient  form  '■ 

0(0)  =  0^"'"exp{-(l-0)s}.  (13) 

The  condition  that  X^^^  >  0  is  satisfied  provided  that 

s>  (m-l)k.  k  =  -(l-0)"^log  0.  (14) 

It  can  be  shown  that,  as  0  increases  from  0  to  1 ,  k=k{0)  decreases  from  to  1 . 

Hence,  the  formula  (13)  is  valid  for  sufficiently  large  values  of  9,  provided 

that  s  >  m-1.  It  holds  whenever  it  produces  a  significance  level  a(0)  <  1  and, 

otherwise,  it  should  be  replaced  by  a{0)  =  1. 

Our  results,  so  far,  depend  on  choosing  a  single  value  of  0  and  we  must 

now  consider  the  general  hypothesis  Hj  represented  by  (7)  for  0  <  0  <  1.  It  is 

easily  verified  that  the  inequality  (9),  which  characterises  stopping  points 

where  the  likelihood  ratio  exceeds  X  .  .  is  equivalent  to 

obs 

-  s  >  (n-m)k(0).  (15) 

This  shows  how  our  ordering  of  the  set  of  possible  results  depends  on  0.  There 
is  no  single  ordering  which  adequately  represents  the  composite  alternative  H^. 
However,  it  will  be  useful  to  determine  the  range  of  values  achieved  by  a(0) 


t 


10 

for  any  Riven  m  and  s. 

By  differentiating  (13),  we  find  that  a' {B)/a(9)  =  s-(m-l)0  so  there  is 

M  ^  X  ^ 

a  unique  minimum  of  a(0)  at  0=0  ={m-l)s  and  0  <1  if  s  >  m-1.  The  case  0  <  s 
<  m-1  is  trivial,  since  it  implies  that  a(0)  =  1,  0  <  0  <  1.  Let  us  assume, 
from  now  on,  that  s  >  m-1.  Then  the  minimum  level  a  =  a(0  )  <  1  and 
a**  =  {"p}"*  ^exp(m-l-s),  s  >  m-1.  (16) 

It  is  also  clear  tliat  any  level  in  the  interval  (a  .1)  can  be  attained  by 
choosing  an  appropriate  0  e  (0  ,1). 


4.  NUMERICAL  ILLUSTRATION 


m 

Finally,  we  can  compiare  the  formulae  (2)  and  (16).  As  before,  s  =  -2  lov, 

1 


u^ ,  where  Uj,U2.--u^  are  the  observed  significance  levels  in  m  independent 
tests  of  the  same  hypothesis.  The  nominal  combined  level  of  significance  Qq 
represents  these  results,  assuming  that  m  was  fixed  in  advance.  On  the  other 
hand,  a  is  a  measure  of  the  evidence  against  the  null  hypothesis  when  no 
stopping  rule  was  prescribed.  The  previous  arguments  can  be  summarised  by 
saying  that  a  is  a  lower  bound  in  the  following  sense  :  it  is  the  smallest 
significance  level  that  can  be  justified  by  a  sequential  likelihood  ratio  test, 
for  a  suitable  alternative  hypothesis. 

It  is  clear  from  (2)  and  (16)  that  =  a  if  m=l .  For  each  m  >  2  and  any 
s  >  m-l,  we  have  0  <  <  a  <1.  This  is  implicit  in  our  construction  of  a  , 

but  it  can  also  be  verified  directly  from  the  formulae.  For  convenience,  we 
shall  consider  the  ratio  a  /^q.  It  is  a  straightforward  exercise  to  establish 

the  following  properties  by  using  (2)  and  (16):  both  and  a  are  decreasing 

w 

in  s  and  a  /a^  is  increasing  in  s,  with 

w 


lim  -2—  =  {-^}"’~^(m-l)!  .  (17) 

„  a„  'm-1^  ' 

s-»  ®  0 

w  w 

When  s  =  m-1.  a  =1  and  <  1 ,  which  confirms  that  a  /a^  >  1  for  all  s  >  m-1. 
The  limit  given  by  (17)  is  an  upper  bound.  The  table  below  shows  values  of  the 
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ratio  a  /a^  for  m  =  2,3 . 10.20.  with  s  chosen  so  that  =  .01.  .TOl, 

final  row  gives  the  limits  as  s->  <»  amd  -*  0. 


TABLE  I 

Ratios  of  significance  levels  :  a  /a 


m 

2 

3 

4 

5 

6 

7 

8 

9 

10 

20 

ao=.01 

2.36 

2.92 

3.26 

3.52 

3.72 

3.89 

4.02 

4.  12 

4.22 

4.81 

V-Ooi 

2.45 

3.09 

3.53 

3.82 

4.08 

4.30 

4.46 

4.61 

4.75 

5 . 56 

a_=  0 

2.72 

3.70 

4.46 

5.  12 

5.70 

6.23 

6.71 

7.16 

7.59 

10.97 
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